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PRACTICAL  THINKING:  REVIEW  OF  COGNITIVE  INSTRUCTION  PROGRAMS 

FOR  BATTLE  COMMAND 

Requirement 

In  August  1994  the  Commander  of  the  Training  and  Doctrine  Command,  GEN 
Frederick  M.  Franks,  Jr.,  requested  that  ARI  play  a  pivotal  role  in  developing  a  course  for  mid¬ 
career  Army  officers.  He  asked  that  instruction  on  thinking,  reasoning,  and  deciding  be 
developed  for  inclusion  in  a  course  on  the  art  of  Battle  Command.  In  the  subsequent  four 
months  the  concept,  content,  and  integration  of  the  instruction  into  the  Command  and  General 
Staff  Officers  Course  were  completed  by  ARI  researchers. 


Goal 

A  search  of  previous  cognitive  instruction  programs  was  instrumental  in  preparing  for 
instructional  development.  The  search  and  review  of  the  programs  was  important  to  provide 
information  on  the  empirical  support  for  cognitive  skill  development  approaches.  More 
importantly  the  review  of  previous  programs  was  useful  to  focus  areas  where  cognitive  skills  are 
most  needed  to  be  improved  and  most  amenable  to  improvement.  The  review  was  also  seen  as 
a  source  of  lesson  materials  to  be  included  in  the  course.  This  report  documents  the  findings 
from  the  review  of  cognitive  instruction  programs  with  these  uses  in  mind. 


Background 

The  current  interest  in  cognitive  instruction  in  critical  thinking  has  arisen  from  evidence 
of  the  current  lack  of  thinking  ability  among  students,  according  to  Idol  and  Jones  (1991).  This 
has  resulted  in  a  movement  where  educators  have  been  asked  to  promote  critical  thinking  with 
cognitive  instruction  in  the  classroom.  Cognitive  instruction  refers  to  any  effort  in  teaching  or 
designing  instructional  materials  to  help  students  process  information  in  meaningful  ways  and  to 
become  independent  learners,  encompassing  problem  solving,  decision  making,  critical  and 
creative  thinking,  memory,  expert  teaching,  and  metacognition  (Idol  &  Jones,  1991).  One  overall 
goal  in  cognitive  instruction  is  to  help  students  learn  how  to  learn.  This  refers  "not  only  to 
independent  application  of  specific  strategies  but  also  to  self-appraisal  and  self-regulation  of  the 
process  of  learning"  (p  70).  The  definition  and  goal  of  cognitive  instruction  contrasts  with 
traditional  instruction  which  focuses  on  content,  and  may  or  may  not  consider  transfer  and 
application  of  specific  skills. 

There  has  been  an  ongoing  debate  in  the  educational  literature  on  whether  to  teach 
thinking  skills  in  a  separate  or  an  integrated  curriculum,  and  whether  or  not  thinking  skills 
taught  separately  will  transfer  or  generalize  to  relevant  contexts  (Idol  &  Jones,  1991,  p.  443).  In 
their  book.  The  Teaching  of  Thinking,  Nickerson,  Perkins,  and  Smith  (1985)  respond  to  the 
debate  by  stating  that  "many  students  do  not  acquire  the  ability  to  think  very  effectively  as  a 
consequence  of  their  educational  experience,  and,  until  recently,  relatively  little  attention  has 
been  given  to  the  possibility  of  mak^g  the  teaching  of  thinking  skills  a  primary  educational 
objective,  in  the  sense  in  which  the  teaching  of  reading,  writing  and  mathematics  are  primary 
educational  objectives"  (p  6).  Voss,  Perkins,  and  Segal  (1991)  state  that  although  schools  do  not 
teach  cognitive  instruction  as  a  separate  subject,  students  are  assumed  to  receive  instruction  in 
reasoning  as  a  byproduct  of  instruction  in  other  subjects;  yet  when  examined  further,  these 
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courses  do  not  necessarily  make  challenging  cognitive  demands  on  students.  Since  Army 
officer’s  leadership  development  programs  have  not  emphasized  to  any  degree  cognitive 
competence  (Kluever,  Lynch,  Matthies,  Owens,  &  Spears,  1992),  practical  thinking  skills  are 
explored  in  this  program. 

The  ability  to  think  clearly  is  a  special  concern  in  Battle  Command.  While  tactical 
commanders  and  staff  officers  have  the  ability  to  think  clearly,  they  may  not  apply  that  ability  to 
the  fullest  extent  possible  due  to  the  stress,  immediacy,  and  complexity  of  the  dynamic  problem 
situations  that  face  them.  It  is  assumed  that  explicit  instruction  can  promote  better  thinking 
under  these  conditions.  The  cognitive  skill  requirements  of  Battle  Command  can  be  made  more 
explicit  by  focusing  on  them  and  incorporating  them  into  instruction.  The  purpose  of  the  review 
was  to  determine  what  has  been  learned  from  past  programs  that  would  apply  to  the 
development  of  cognitive  instruction  for  mid-career  Army  officers. 


Cognitive  Instruction  Programs 

The  following  summary  describes  fifteen  cognitive  instruction  programs.  Candidate 
programs  were  identified  from  literature  searches.  For  programs  to  be  selected  they  had  to 
have  substantial  thought  and  effort  behind  them.  Review  emphasis  was  placed  on  program 
evaluation,  usefulness,  adaptability,  and  predictive  outcome.  The  programs  were  analyzed  by 
considering  nine  different  questions: 

1.  What  are  the  underlying  assumptions  of  the  program? 

2.  What  are  the  overall  goals  of  the  program? 

3.  What  types  of  methods  and  materials  are  used  in  the  program? 

4.  What  are  the  specific  thinking  skills  taught  in  the  program? 

5.  What  is  the  program’s  target  population? 

6.  What  is  the  expected  outcome  of  the  program? 

7.  Do  the  activities  promote  independent  generalized  learning? 

8.  What  types  of  activities  are  learners  engaged  in? 

9.  What  evidence  of  skill  acquisition  is  available? 

Of  these  questions,  promotion  of  independent  generalized  learning  (#7)  is  probably  the 
most  important.  If  students  are  unable  to  apply  the  concepts  taught  and  in  appropriate  new 
learning  situations,  then  the  effort  devoted  to  direct  instruction  is  wasted.  Tliis  is  also  one  of 
the  most  difficult  to  assess  because  it  is  difficult  just  to  measure  thinking  performance  within  the 
benign  conditions  of  the  classroom,  let  alone  the  more  complex  situations  of  the  world  and 
learning  over  time. 

Descriptions  of  the  programs  are  ordered  chronologically.  They  include  instructional 
topics  dealing  with  attitudes,  cognitive  skill,  tools,  and  metacognition.  Tables  1  through  3 
summarize  the  concept  and  targeted  skills  of  each  of  the  programs. 


2 


Table  1. 

Description  of  cognitive  instruction  programs  (1967-1976). 
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Table  2. 

Description  of  cognitive  instruction  programs  (1979-1984). 
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Table  3. 

Description  of  cognitive  instruction  programs  (1985-1992). 
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Structure  of  the  Intellect  (SOI)  (Guilford,  1967;  Guilford  &  Hoepfner,  1971);  extended  by 
Meeker  (1969)  into  a  design  for  educational  application. 

Assumptions:  Based  on  Guilford’s  model  of  intelligence,  SOI  is  a  diagnostic  and 
prescriptive  measure  of  intelligence  described  in  terms  of  the  intersection  of  five  mental 
operations,  four  contents,  and  six  products.  It  is  assumed  that  all  students  have 
intelligence  and  the  SOI  program  answers  "What  kind?"  rather  than  "How  much?" 

Goals:  To  lay  the  basic  foundation  for  sequencing  learning  abilities.  Once  basic  abilities 
are  mastered,  higher-level  reasoning  or  critical  thinking  abilities  are  more  easily 
assimilated.  The  overall  goal  is  to  equip  students  with  the  necessary  intellectual  skills  to 
learn  subject  matter  and  critical  thinking. 

Methods  and  materials:  The  method  is  straightforward.  First,  students  take  a  battery  of 
diagnostic  SOI  tests,  usually  the  SOI-LA  (learning  abilities)  tests.  The  battery  consists  of 
items  within  each  of  the  following  categories:  cognition,  memory,  evaluation  (judging, 
planning,  decision  making,  analysis,  reasoning  and  logic),  convergent  production 
(problem  solving),  and  divergent  production  (creative  problem  solving).  Finally,  after  the 
results  are  available,  suggestions  are  made  on  how  to  improve  weaknesses. 

Thinking  skiUs:  Guilford’s  three  dimensional  model  of  the  SOI  is  classified  into  three 
general  categories  with  specific  sub-areas:  (1)  Operations  -  cognition,  memory, 
convergent  and  divergent  thinking.  (2)  Content  -  figural,  semantic,  symbolic,  behavioral. 
(3)  Products  -  units,  classes,  relations,  systems,  transformations,  implications. 

Targeted  population:  preschool  to  adult. 

Expected  outcome(s):  After  diagnostic  tests  indicate  areas  of  intelligence  that  are  weak, 
"prescriptions"  are  made  which  are  expected  to  positively  affect  the  weaknesses. 

Promotion  of  independent  generalized  learning:  There  are  no  readily  available  reports 
of  direct  measures  of  the  impact  of  the  training  or  general  school  performance. 

Example:  If  a  pupil  is  found  to  be  weak  in  divergent  production,  the  following  may  be 
prescribed:  Teachers  and  parents  should  work  with  the  student  to  reclassify  objects, 
pictures,  numbers  and  words  used  in  unusual  ways,  and  brainstorm  ideas  and  write 
creatively. 

Efficacy  evidence:  There  is  fair  amount  of  research  regarding  the  effectiveness  of  the 
SOL  Broadly,  evidence  indicates  that  intelligence  can  be  trained  and  result  in  an 
improved  self-concept  and  improved  attitudes  towards  school.  Specifically,  one  study 
indicated  that  the  SOI  test  was  effective  in  assisting  educators  in  diagnosing  and  aiding 
students  toward  greater  school  achievement.  Another  study  reported  that  there  were 
significant  gains  in  SAT  assessments  of  arithmetic  but  not  reading  ability.  One  source 
indicated  that  "the  SOI  program  improves  intelligence"  (Nickerson,  Perils,  &  Smith, 
1985,  p  168). 
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Cognitive  Research  Trust  Thinking  (CoRT)  (de  Bono,  1973,  1976,  1991) 


Assumptions:  There  are  two  stages  in  thinking:  perception  and  analysis.  Poor  thinking  is 
often  due  to  errors  in  perception,  rather  than  analysis.  Perceptual  skills  are  neglected  by 
schools  and  therefore  deserve  greater  attention  in  thinking  programs. 

Goals:  Although  analytic  thinking  is  covered,  the  emphasis  is  on  improving  perceptual 
thinking  through  the  use  of  "tools,"  such  as  PMI  (Plus,  Minus,  Interesting)  and  C&S 
(Consequences  and  Sequels),  which  draw  attention  to  the  perceptual  stage  of  thinking. 

Methods  and  materials:  Practice  in  use  of  perceptual  tools  on  real-life  problems  through 
group  and  class  discussion.  One  fast  paced  lesson  per  week  for  two  years. 

Thinking  skills;  Breadth  -  how  to  broaden  thinking;  organization  -  how  to  organize 
thinking;  interaction  -  how  to  assess  evidence;  creativity  -  how  to  escape  from 
imprisoning  ideas;  and  action  -  concerned  with  the  total  process  of  thinking. 

Tools  are  the  thinking  skills  that  students  practice  in  CoRT  lessons.  The  CoRT  tools 
have  deliberately  strange-sounding  names  in  order  to  make  them  separate  from  content 
as  attention-directors.  The  tools  are  practiced  on  a  wide  range  of  situations  described  on 
student  work  cards.  Each  situation  is  only  considered  for  a  short  burst  of  time  (2-4 
minutes).  Then  the  student  applies  the  tool  to  another  matter.  This  process  is  purported 
to  be  most  effective  in  keeping  attention  on  the  tool  rather  than  on  the  content.  Once 
the  tool  is  mastered,  the  skill  can  theoretically  be  applied  to  mastery  of  content  in 
various  subjects. 

PMI:  P  stands  for  Plus  (the  good  points),  M  stands  for  Minus  (the  bad  points),  I 
stands  for  the  Interesting  points  about  a  problem.  The  interesting  points  raise  what  if 
questions.  PMI  encourages  students  to  look  at  different  aspects  of  the  situation 
instead  of  making  an  initial  judgment.  The  explanation  following  the  PMI  description 
includes  techniques  for  generating  alternatives,  being  perceptive,  identifying  patterns, 
and  using  humor. 

CAP  (Consider  All  Factors)  -  CAP  involves  an  explicit  listing  of  advantages  and 
disadvantages.  Designed  to  counter  the  supposed  natural  tendency  of  humans  to  be 
egocentric. 

C&S  (Consequence  and  Sequel)  -  Designed  to  assist  the  student  in  explicitly  thinking 
of  long  term  possibilities,  as  opposed  to  the  usual  impulsive  response. 

EBS  (Examine  Both  Sides)  -  Students  are  encouraged  to  take  the  opposing  side  in  an 
argument  and  develop  it,  even  beyond  the  actual  view  of  the  other  person. 

ADI  (Agreement,  Disagreement  and  Irrelevance)  -  ADI  directs  attention  to  areas  of 
agreement,  disagreement,  and  irrelevance.  Judgment  about  what  makes  up  these 
areas  depends  on  the  student. 

OPV  (Other  People’s  Views)  -  The  student  is  encouraged  to  consider  the  points  of 
view  of  aU  those  involved  and  express  them. 
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HV/LV  (High  values/low  values)  -  Values  are  divided  into  high  and  low  to  encourage 
a  closer  examination  of  the  values  involved  in  a  situation. 

AGO  (Aims,  Goals,  and  Objectives)  -  Students  are  shown  to  resist  the  temptation  to 
discriminate  between  aims,  goals,  and  objectives  unless  a  particular  situation  lends 
itself  to  the  use  of  a  particular  term. 

TEC  (Target  and  Task,  Expand  and  Explore,  Contract  and  Conclude)  -  This  tool  is 
meant  to  encourage  more  deliberate  thinking. 

Targeted  population:  ages  9  to  12. 

Expected  outcome(s):  To  teach  lateral  thinking  skills  (i.e.,  learning  to  restructure  the 
problem  space)  as  opposed  to  vertical  or  logical  thinking  (i.e.,  sequential,  predictable, 
and  unconventional). 

Promotion  of  independent  generalized  learning:  Once  students  master  the  tools,  they 
are  encouraged  to  use  them  in  all  subjects.  Transfer  to  other  domains  and  subject  matter 
is  expected  to  occur  spontaneously. 

Example:  The  students  are  asked  by  the  teacher  whether  they  would  like  to  be  paid  five 
dollars  a  week  for  attending  school.  Working  in  small  groups,  the  students  list  aU  of  the 
good  points  about  the  idea,  then  the  disadvantages,  and  finally  the  features  that  are  not 
easily  classified  as  either  plus  or  minus,  but  are  worthy  of  notice. 

Efficacy  evidence:  Much  of  the  research  regarding  the  effectiveness  of  CoRT  is  reported 
in  anecdotal  form,  but  little  is  available  regarding  statistical  analysis  of  improved 
performance  on  specific  tests  of  reasoning  or  correlated  with  increased  standardized 
achievement  tests.  A  study  by  de  Bono  (1976)  revealed  that  CoRT  trained  students 
tended  to  give  more  attention  to  all  sides  of  an  issue,  considering  general  as  well  as 
personal  consequences.  The  issue  of  transfer  of  CoRT  skills  into  other  subjects, 
according  to  de  Bono,  depends  largely  upon  whether  the  teacher  reminds  students  to  do 
so. 


Productive  Thinking  Program  (Covington,  Crutchfield,  Davies,  &  Olton,  1974) 

Assumptions:  Productive  thinking  involves  five  kinds  of  thinking  skills.  Improvement  in 
these  skills  may  be  accomplished  through  instruction  in  general  strategies  ("thinking 
guides").  Direct  instruction  and  practice  are  essential.  Development  of  good  attitudes  is 
fundamental  to  success. 

Goals:  To  teach  the  use  of  sixteen  thinking-guides  that  are  helpful  in  productive 
thinking. 

Methods  and  materials:  Students  read  fifteen  booklets  in  comic  book  format,  and 
practice  thinking  by  answering  questions  and  confronting  story  characters.  Booklet 
exercises  are  discussed  by  the  class.  The  program  is  usually  taught  in  one  semester. 

Thinking  skills:  Thinking  productively  requires  the  use  of  reasoning  and  critical  analysis 
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but  also  imagination  and  creativity.  It  involves  five  kinds  of  skill:  discovering  and 
formulating  problems,  organizing  and  using  information,  generating  ideas,  evaluating  and 
improving  ideas,  and  creating  new  perspectives. 

Targeted  population:  primarily  fifth-  and  sixth-grade  students;  suitable  for  all  students 
except  the  slowest  learners. 

Expected  outcome(s):  Students  are  expected  to  use  more  original  ideas,  persist  at 
problem  solving,  have  curiosity,  and  use  thinking-guides  in  attacking  problems. 

Promotion  of  independent  generalized  learning:  Some  exercises  show  students  how  the 
thinking-guides  apply  to  their  regular  schoolwork.  For  example,  in  one  exercise  students 
are  asked  to  identify  the  guides  that  might  be  used  in  writing  an  essay. 

Example:  In  the  first  lesson,  the  two  principle  characters  enter  a  map-making  contest. 
One  character  is  unable  to  come  up  with  any  ideas  at  all,  and  the  other  character  has 
ideas  but  lacks  the  confidence  to  share  them.  The  characters’  reactions  should  be 
familiar  to  the  children  and  serve  to  show  them  that  many  others  share  their  fears  and 
frustrations  when  faced  with  problems. 

Efficacy  evidence:  Research  on  the  effectiveness  on  this  program  has  yielded  mixed 
findings.  Sometimes  the  program  produced  significant  gains  on  a  range  of  posttests,  and 
sometimes  it  did  not.  One  study  noted  that  the  greatest  successes  seem  to  have  been 
achieved  under  the  least  well  controlled  conditions;  they  suggest  that  the  program  might 
be  most  effective  when  applied  in  a  small  group  with  an  enthusiastic  teacher. 

When  testing  for  transfer  to  similar  problems,  performance  was  measured  in  several 
ways:  ultimate  success  or  failure,  fluency  in  generating  options,  quality  of  ideas,  and 
anomalies  noticed  in  the  data  that  may  point  to  a  solution.  The  findings  of  three 
experiments  significantly  favored  the  treatment  group  over  the  control.  However,  on 
individual  items,  sometimes  one  measure  of  performance  had  an  advantage  over  another 
(e.g.,  more  fluency,  better  quality  ideas,  noticing  anomalies,  but  without  solving  the 
problem  more  frequently). 

Regarding  divergent  thinking,  the  various  ideational  fluency  instruments  used  in  the 
evaluations  did  not  show  gains  in  the  treatment  groups  relative  to  the  control  group.  For 
problems  and  tasks  of  a  markedly  different  character,  there  is  no  clear  evidence  that  the 
program  helps.  However,  it  has  not  been  tested  how  well  the  program  might  prepare 
students  to  deal  with  complex  problems  on  their  own. 


Patterns  of  Problem-Solving  (Rubenstein,  1975,  1980) 

Assumptions:  What  one  needs  in  order  to  be  an  effective  thinker  is  a  variety  of 
problem  situations,  along  with  metaknowledge  about  situations  in  which  specific 
heuristics  are  appropriate. 

Goals: 

1.  To  develop  a  general  foundation  of  problem  solving  approaches  and  master  some 
techniques. 
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2.  To  provide  a  foundation  for  attitudes  and  skUls  productive  in  dealing  with  problems  in 
the  context  of  human  values. 

3.  To  emphasize  the  thinking  process  at  all  stages  of  the  problem  solving  activity. 

4.  To  identify  individual  problem  solving  styles  and  learn  to  overcome  conceptual  blocks 
and  self-imposed  constraints. 

5.  To  expose  students  to  both  objective  and  subjective  aspects  of  problem  solving. 

6.  To  provide  a  framework  for  a  better  appreciation  of  the  role  of  tools  and  concepts 
acquired. 

7.  To  bring  together  students  from  diverse  backgrounds  so  that  they  can  observe 
different  attitudes  and  problem  solving  styles  and  learn  from  each  other.  (Rubenstein, 
1980,  p  26) 

Methods  and  materials:  The  text,  Patterns  of  Problem  Solving  (Rubenstein,  1975),  is 
designed  to  provide  the  reader  with  tools  and  concepts  that  are  productive  in  problem 
solving,  and  are  least  likely  to  erode  over  time.  The  two  courses  at  UCLA  have  come 
from  the  first  and  second  portions  of  the  text. 

Thinking  skills:  Metacognition,  problem  representation,  course  of  action  selection  and 
implementation,  representation  transformation,  restatement  of  the  question,  challenging 
of  premises  and  assumptions,  using  analogies  and  metaphors,  discussing  the  problem 
with  others,  using  partial  solutions,  following  hunches  and  attending  to  feelings. 

Targeted  population:  college  students. 

Expected  outcome(s):  To  improve  everyday  problem  solving  by  using  concrete 
representation  and  systematically  considering  many  alternative  courses  of  action. 

Example:  Use  of  diagrams  for  representing  problems,  math  word  problems,  probability 
problems,  etc. 

Efficacy  evidence:  A  general  intelligence  (IQ)  test  (California  test  of  Mental  Maturity, 
Short  Form  1963)  was  given  to  students  before  and  after  taking  the  college  course  in 
Patterns  of  Problem  Solving.  Rubenstein  (1980)  reports  an  average  improvement  of  5.4 
scale  points  in  IQ. 


Philosophy  for  Children  (Lipman,  1976) 

Assumptions:  Thinking  well  requires  the  ability  to  perform  numerous  reasoning  skills, 
most  of  which  are  best  learned  through  the  use  of  language  (i.e.,  dialogue). 

Goals:  To  teach  students  to  reason  well  and  to  enjoy  thinking  for  themselves. 

Methods  and  materials:  Students  meet  three  times  a  week  for  forty  minutes  to  read,  do 
exercises,  and  talk.  The  focus  of  the  class  is  a  novel  in  which  the  characters  discover  and 
model  principles  of  reasoning  in  the  process  of  exploring  philosophical  issues.  Teachers 
use  a  variety  of  special  techniques  to  model  and  elicit  reasoning  skills. 

Thinking  skills:  Inquiry,  discovery  and  invention,  inference,  inductive  reasoning,  styles  of 
thinking,  generalization,  contradiction,  possibilities,  cause  and  effect,  explanations  and 
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descriptions,  and  considering  consequences  when  deciding  what  to  do. 

Targeted  population:  grades  3  to  12. 

Expected  outcome(s):  To  help  students  think  for  themselves,  to  explore  alternatives  to 
their  own  point  of  view,  to  consider  evidence  more  logically  and  objectively,  and  to 
search  for  presuppositions  and  reasons  for  their  beliefs. 

Promotion  of  independent  generalized  learning;  Practical  applications  of  the  principles 
of  logic  that  are  being  studied  are  emphasized,  and  practice  is  provided  in  analyzing 
logically  such  things  as  advertisements,  letters  to  the  editor,  and  political  cartoons. 

Example:  An  important  philosophical  exercise  is  that  of  uncovering  the  presuppositions 
on  which  questions  and  statements  are  based.  Teachers  can  help  students  search  for 
assumptions  by  asking  them,  "Are  you  assuming  that...?  or  Doesn’t  what  you  say  depend 
upon  the  assumption  that...?" 

Efficacy  evidence:  In  a  Rutgers  University  study,  a  comparison  of  reading  skills  on  the 
Metropolitan  Achievement  test  revealed  that  students  who  had  gone  through  the 
program  gained  an  average  of  eight  months  in  reading  ability,  while  a  control  group 
gained  five  months.  The  program  took  80  classroom  hours  to  achieve  the  3  month  gain 
in  reading  ability. 


Heuristic  Instruction  in  Mathematical  Problem  Solving  (Schoenfeld,  1980) 

Assumptions;  Heuristics  help  students  solve  problems  when  the  students  know  and  apply 
the  heuristics,  but  students  lack  a  good  set  of  heuristics.  Students  do  not  reliably  pick  up 
heuristics  spontaneously  from  examples;  heuristics  have  to  be  explicitly  taught.  Students 
do  not  reliably  apply  heuristics  they  know  about;  some  sort  of  guidance  or  prompting  is 
necessary.  A  "managerial  strategy"  for  approaching  problems,  taken  together  with 
heuristics,  can  help  students  to  apply  heuristics  and  lead  to  substantially  improved 
problem  solving  performance  in  mathematics. 

Goals:  To  not  only  teach  the  use  of  heuristics  in  solving  math  problems,  but  to  teach 
them  in  a  "managerial"  manner.  In  other  words,  once  a  problem  is  attacked,  the  problem 
solver  will  continue  to  monitor  his  progress  by  applying  heuristics  throughout  the 
process. 

Methods  and  materials:  Training  occurs  in  the  following  five  phases: 

1.  Analysis  -  once  a  problem  is  given,  the  aim  is  to  get  a  "feel"  for  the  givens  and 
unknowns.  Another  aim  is  to  simplify  the  problem  using  heuristics. 

2.  Design  -  to  develop  a  plan  for  proceeding  and  ensure  that  detailed  calculations  are  not 
done  prematurely;  no  specific  heuristics  are  suggested. 

3.  Exploration  -  Chosen  when  the  problem  presents  difficulties  and  no  clear  plan  for 
directly  producing  a  solution  is  at  hand;  several  heuristics  are  suggested. 

4.  Implementation  -  used  when  a  plan  is  in  hand  that  should  lead  to  a  solution  if  carried 
through;  no  heuristics  are  suggested. 

5.  Verification  -  to  check  the  solution;  several  heuristics  are  suggested. 
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Thinking  skills:  The  trained  use  of  heuristics  in  a  "managerial"  manner. 

Targeted  population:  college  students. 

Expected  outcome(s):  To  significantly  impact  in  a  positive  manner  the  math  problem 
solving  skills  of  the  math  student. 

Promotion  of  independent  generalized  learning:  There  was  no  information  addressing 
this  in  the  literature  reviewed. 

Example:  During  the  analysis  phase,  the  problem  solver  may  try  to  diagram  the 
problem,  in  order  to  simplify  it. 

Efficacy  evidence:  Experimental  evidence  suggests  that  the  instruction  results  in 
substantial  improvements  in  math  problem  solving. 


A  Practicum  In  Thinking  (Steiner.  1979) 

Assumptions:  Steiner’s  motivation  for  developing  a  course  followed  from  the  conviction 
that  "in  general,  our  education  system  does  not  teach  its  students  how  to  think,  nor  does 
it  seek  directly  to  do  so"  (Steiner,  1979,  p.3). 

Goals:  This  approach  has  a  distinctly  metacognitive  aspect.  Emphasis  is  placed  on  self- 
awareness.  Students  are  encouraged  to  introspect  on  how  they  are  performing  the  tasks. 
The  purposes  of  the  project  are  (1)  to  get  the  student  to  learn  something  about 
themselves  by  paying  attention  to  how  they  go  about  solving  the  problem,  (2)  to  make 
them  aware  of  the  element  of  choice  in  strategizing  for  problem  solving,  and  (3)  to  teach 
them  the  value  of  approaching  a  problem  systematically. 

Methods  and  materials:  The  class,  the  size  of  which  is  limited  to  about  25  students, 
meets  for  2  two-hour  sessions  per  week  for  10  weeks.  It  is  divided  into  four  groups  of 
roughly  equal  size  that  stay  together  throughout  the  10-week  period.  Class  sessions  are 
usually  devoted  to  exercises,  each  of  which  addresses  a  specific  goal.  Each  exercise  ends 
with  a  set  of  questions  for  discussion  to  help  the  students  analyze  what  they  have  learned 
in  doing  it. 

Thinking  skills:  Topics  include  the  following:  working  in  groups,  listening,  words  and 
meanings,  assumptions,  study  skills,  memory,  problem  analysis,  logical  inference,  problem 
solving,  decision  making,  and  creative  problem  solving. 

Targeted  population:  college  students 

Expected  outcome(s):  To  make  students  realize  that  most  problems  can  be  approached 
in  more  than  one  way,  and  to  make  them  conscious  of  the  fact  that  they  have  some 
options  in  that  regard. 

Promotion  of  independent  generalized  learning:  There  was  no  information  addressing 
this  in  the  literature  reviewed. 
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Example:  Students  are  required  to  work  on  a  problem  of  their  own  choosing  as  a  special 
course  project,  for  example  deciding  about  graduate  school. 

Efficacy  evidence:  Attempts  to  evaluate  the  effectiveness  of  the  course  have  included 
self-evaluations  by  the  students  before  and  after  the  course,  with  respect  to  certain 
specific  skills  (working  with  others,  memory,  and  generating  new  ideas).  Students 
considered  themselves  to  be  more  adept  after  the  course  than  before  the  course.  Even  if 
there  were  no  actual  gains  in  skill,  this  enhanced  self-perception  could  have  a  positive 
effect  on  the  students’  handling  of  problems  (Wheeler  &  Dember,  1979). 


Instrumental  Enrichment  Program  (Feuerstein,  1980;  SaveU,  Twohig,  &  Rachford,  1986) 

Assumptions:  Students  learn  to  think  by  means  of  "mediated  learning  experiences."  A 
mediated  learning  experience  is  when  someone  other  than  the  learner  or  something  else 
(e.g.,  computer  assisted  instruction)  directs  attention  to  salient  teaching  points.  A  lack  of 
such  experiences  may  result  in  "cognitive  deficiencies." 

Goals:  To  correct  cognitive  deficiencies  such  as  impulsivity  and  lack  of  appreciation  for 
accuracy;  to  replace  a  passive  orientation  with  an  active  one.  The  six  subgoals  of  the 
program  include:  (1)  correction  of  deficient  cognitive  functions,  (2)  acquisition  of  certain 
basic  concepts,  labels,  vocabulary,  operations,  and  relationships  necessary  for  cognitive 
tasks,  (3)  production  of  intrinsic  motivation,  (4)  production  of  reflective,  insightful 
thinking,  (5)  creation  of  task-intrinsic  motivation,  and  (6)  instillation  in  the  self  as  an 
active  generator  of  knowledge  and  information. 

Methods  and  Materials:  Fifteen  units  consisting  of  paper  and  pencil  exercises  designed 
to  correct  cognitive  deficiencies.  One-hour  classes  meet  three  to  five  times  a  week  for 
two  to  three  years.  Teacher  leads  discussion  of  exercises.  Eighty  hours  or  more  are 
required  over  a  1  to  3  year  period. 

Thinking  skills:  The  skills  include  categorization,  orientation  in  space,  recognizing 
relationships,  following  directions,  planning,  organizing,  logical  reasoning,  and 
^thesizing. 

Targeted  population:  ages  11  to  adult. 

Expected  outcome(s):  To  correct  cognitive  deficiencies  such  as  impulsivity  and  inaccuracy 
in  student  responses  to  learning. 

Promotion  of  independent  generalized  learning:  Students  are  required  to  work 
independently  with  emphasis  placed  on  teacher  mediation  of  feelings  of  competency  and 
on  developing  and  maintaining  motivation;  skills  are  taught  within  isolated  contexts;  then 
bridging  activities  are  presented  that  are  intended  to  expedite  transfer. 

Example:  One  lesson  involves  comparisons,  in  which  the  student  is  required  to  look  for 
similarities  and  differences  between  two  or  more  objects  or  concepts.  The  dimensions 
examined  are  sometimes  concrete  (number,  color,  form)  and  sometimes  abstract 
(function,  composition,  power).  In  another  exercise,  the  student  ranks  five  figures 
according  to  how  closely  they  resemble  a  model. 
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Efficacy  evidence:  Twohig,  Rachford,  Saveli,  and  Rigby  (1987)  state  that  where  the 
program  has  been  implemented  thoroughly  and  reliable  measures  have  been  used,  there 
has  been  an  improvement  in  cognitive  performance.  Typically,  there  is  no  evidence  for 
statistically  significant  results  until  well  into  the  program  (close  to  100  hours  of  training). 
Feuerstein  (1980)  found  that  students  who  had  gone  through  the  program  continued  to 
improve  in  cognitive  performance,  relative  to  control  groups  two  years  after  the  program 
ended. 


Heuristic  Model  (Friedman,  1984) 

Assumptions:  Content  mastery  is  becoming  less  important  because  today’s  information 
quickly  becomes  obsolete  but  the  skills  necessary  for  processing  information  are 
becoming  more  important.  Heuristics  help  one  learn  by  providing  a  method  for  learning 
and  investigation  that  can  be  applied  to  a  wide  range  of  content  or  situations.  Thinking 
skills  taught  directly  is  superior  to  incidental  learning. 

Goals:  Teach  prediction  concurrently  with  supporting  thinking  skills  in  a  combined 
process. 

Methods  and  Materials:  Teachers  choose  a  strategy  based  on  assessing  student 
knowledge  or  ability  for  a  specific  area,  such  as  the  American  Civil  War.  The  teacher 
may  choose  homework  or  classroom  instruction  to  remedy  the  deficiency.  The  teacher 
plans  how  to  teach  each  basic  skill  in  the  context  of  the  Civil  War. 

Thinking  skills:  identification  (concrete,  categorical,  programmatic,  hierarchical), 
identifying  problems,  predicting  events,  predicting  consequences,  testing  predictions,  and 
revising  predictions. 

Targeted  population:  gifted  learners;  below  the  5th  grade  the  order  of  presentation  is 
modified. 

Expected  outcome:  Students  will  combine  thinking  skills  to  solve  problems. 

Promotion  of  independent  generalized  learning:  The  program  claims  that  teaching  of 
higher  order  thinking  skills  will  lead  to  improved  ability  to  discover  and  invent. 

Example:  For  the  skill  of  predicting  the  consequences  of  solutions,  the  teacher  might 
have  the  students  identify  how  the  Union  thou^t  the  Confederacy  could  be  defeated. 
The  teacher  explores  in  more  depth  what  planning  and  actions  had  to  take  place. 

Efficacy  evidence:  None  given  in  sources  reviewed. 


roEAL  (Bransford  &  Stein,  1984,  1993) 

Assumptions:  People  are  capable  of  figuring  out  a  workable  solution  if  they  think  about 
the  problem.  People  can  learn  to  deal  with  the  problems  that  they  encounter  by  paying 
attention  to  their  approach  to  problem  solving.  This  will  be  especially  important  when 
people  are  faced  with  nonroutine  problems.  A  requirement  for  successful  problem 
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solving  is  effective  learning. 

Goals:  To  show  people  which  problem  solving  processes  that  work  and  which  do  not. 
The  IDEAL  model  is  to  help  improve  problem  solving.  IDEAL  stands  for  Identify 
problems  and  opportunities,  Define  goals,  Explore  possible  strategies.  Anticipate 
outcomes  and  Act,  and  Look  back  and  Learn.  The  "ideal  problem  solver  is  someone 
who  continually  attempts  to  improve  by  paying  attention  to  his  or  her  processes  and  by 
learning  from  any  mistakes  that  are  made."  (p  xiii,  Bransford  &  Stein,  1993). 

Methods  and  materials:  Students  require  frequent  opportunities  for  self-assessment  to 
analyze  the  effects  of  their  actions  and  to  learn  from  them.  Instruction  should  be  based 
on  realistic  problems  that  are  relevant  to  the  needs,  interests,  and  skills  of  the  students. 
The  Problem  Navigation  Guide  can  be  used  to  steer  one  through  the  IDEAL  cycle. 

Thinking  skills:  Students  will  be  able  to:  recognize  potential  problems  and  treat  them  as 
opportunities,  carefully  define  the  goal,  explore  potential  alternative  approaches  for 
solving  the  problem,  anticipate  likely  outcomes  before  acting,  and  examine  the  outcome 
and  its  effects  to  learn  from  the  experience. 

Targeted  population:  The  target  population  is  not  clearly  specified,  but  it  includes  at 
least  adults. 

Expected  outcomes(s):  Existing  knowledge  will  be  used  more  effectively  to  solve 
problems  and  new  information  will  be  learned. 

Promotion  of  independent  generalized  learning:  The  improvement  of  problem  solving 
and  learning  skills  should  be  ongoing  activities. 

Example:  The  problem  of  a  squeaking  door  can  be  solved  relative  to  the  particular  goal 
that  is  adopted.  The  solution  to  the  problem  depends  on  whether  the  goal  is  to  (1) 
eliminate  the  noise,  (2)  keep  the  sound  from  entering  another  room  because  it  disturbs 
the  people  in  the  room,  or  (3)  to  keep  the  sound  from  being  disturbing.  The  goal  that  is 
selected  suggests  particular  strategies  for  solution. 

Efficacy  evidence:  None  given  in  sources  reviewed. 


Critical  Thinking  in  American  History  fCTAHI  (O’Reilly,  1983-1985) 

Assumptions:  Thinking  skills  are  like  athletic  skills  and  should  be  taught  in  a  similar 
way.  For  example,  a  golf  pro  giving  a  lesson  to  a  beginner  breaks  the  swing  down  into  its 
component  parts  and  explains  each  component  while  supervising  the  learner,  who 
practices  for  the  next  lesson.  Thinking  skills  are  unlike  athletic  skills,  however,  in  that  too 
much  instruction  on  which  skills  to  use  may  unnecessarily  limit  student  creativity  and 
insights. 

Goals:  The  goal  of  CTAH  is  informal  reasoning,  or  providing  good  reasons  for  beliefs. 
This  encompasses  skills  such  as  evaluating  evidence,  recognizing  assumptions,  willingness 
to  suspend  judgment,  and  going  beyond  simple  solutions. 
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Methods  and  materials:  The  CTAH  project  uses  four  books,  along  with  associated 
teachers’  guides.  These  materials  are  designed  as  supplementary  resources  for  high 
school  American  History  courses.  The  books  are  made  up  of  historical  problems  and 
interpretations  as  well  as  skills  worksheets  that  identify  a  broad  range  of  informal 
reasoning  skills. 

Thinking  skills:  Identifying  and  evaluating  evidence,  distinguishing  conclusions  from 
premises,  identifying  unstated  assumptions,  identifying  imprecise  words,  identifying 
connections  among  parts  of  an  argument,  and  evaluating  ethical  claims. 

Targeted  population:  high  school. 

Expected  outcome(s):  For  students  to  change  their  view  of  the  nature  of  knowledge,  to 
begin  to  see  it  as  fragmentary,  selective,  and  open  to  interpretation,  and  to  see 
knowledge  as  something  to  be  sought  after  rather  than  something  served  as  so  many 
facts  to  be  memorized. 

Promotion  of  independent  generalized  learning:  After  students  become  skilled  at 
analyzing  other’s  arguments,  they  participate  in  classroom  debates  and  prepare  essays 
presenting  their  views  of  controversial  issues.  Students  learn  how  to  construct  and 
evaluate  their  own  arguments. 

Example:  Students  are  given  the  following  argument  and  asked  to  find  the  ethical  claim: 
"The  U.  S.  was  justified  in  taking  a  strong  stand  against  the  Soviets  in  the  late  1940’s, 
because  the  Soviets  had  broken  their  promise  to  hold  free  elections  in  Eastern  Europe." 

Efficacy  evidence:  O’Reilly  (1983)  states,  "...some  students  have  demonstrated  a 
willingness  to  be  more  self-reflective.  Journal  entries  on  a  decision-making  simulation  on 
the  Vietnam  war  showed  a  great  deal  of  empathy  for  historical  decision  makers...Dealing 
with  assumptions,  overgeneraUzations,  frames  of  reference,  and  so  forth  seems  to  have 
raised  students  consciousness  and  given  them  time  to  examine  their  own  thinking." 
(O’Reilly,  1983,  p  378). 

Intelligence  Applied  (Sternberg,  1986)i 

Assumptions:  This  program  is  based  on  the  "triarchic"  theory  of  intelligence  which 
consists  of  three  parts: 

1.  mental  processes  that  comprise  intelligent  behavior  (metacomponents,  performance 
components,  and  knowledge  acquisition  components). 

2.  real  world  contexts  in  which  intellectual  processes  operate  (skills  for  adapting  to, 
shaping,  and  selecting  environments). 

3.  experience  and  intelligence  (the  role  of  novelty  and  automaticity  in  intelligent 
performance). 

The  theory  proposes  that  intelligence  involves  adapting  to,  shaping  of,  and  selecting 
environments.  Practical  intelligence  deals  with  problems  that  have  no  right  or  wrong 
answer.  Intelligence  differs  depending  on  the  effectiveness  of  response  in  situations 
involving  elements  of  risk,  uncertainty,  and  ambiguity. 
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Goals:  The  program  is  designed  to  help  students  improve  their  abilities  to  perform  the 
processes  assumed  to  underlay  intelligent  behavior. 

Methods  and  materials:  This  can  be  used  as  either  a  semester  or  a  year-long  course.  It 
consists  of  two  elements:  (1)  a  student’s  text  with  narrative  material  and  exercises  for 
students  to  complete,  and  (2)  a  teacher’s  guide  with  material  used  to  maximize  the 
program’s  effectiveness. 

Thinking  skills:  The  information-processing  components  of  intelligence  contain  the 
following: 

1.  Metacomponents  -  defining  problems,  selecting  strategy,  solution  monitoring. 

2.  Performance  components  -  inferring  relations  between  stimuli,  mapping  higher  order 
relations  between  stimuli. 

3.  Knowledge  acquisition  components  -  selective  encoding,  selective  comparison 

Targeted  population:  secondary  school  and  college  students. 

Expected  outcome(s):  Training  is  aimed  at  increasing  students’  abilities  to  identify 
potential  problems  during  the  planning  stages  of  problem  solving,  to  understand  how 
different  reasonable  definitions  of  problems  lead  to  different  strategies,  to  select 
appropriate  strategies,  and  to  monitor  the  effects. 

Promotion  of  independent  generalized  learning:  Skills  and  procedures  that  have  a  great 
deal  of  generality  are  emphasized.  For  example,  in  the  section  on  knowledge  acquisition, 
students  are  taught  to  infer  the  meanings  of  new  words  from  context,  a  skill  most  people 
can  use  throughout  their  lives. 

Example:  For  instruction  on  practical  intelligence,  students  are  presented  with  scenarios 
of  real-world  problems  for  which  they  must  generate  conflict  resolutions. 

Efficacy  evidence:  Sternberg  (1986)  notes  that  the  clinical  data  are  highly  favorable, 
however  statistical  outcome  data  are  not  presented. 


Odvssev  (Harvard  University,  1986)  (Note  that  Project  Intelligence  was  a  related  program.) 

Assumptions:  Intellectual  performance  depends  on  abilities,  methods,  knowledge,and 
attitudes.  These  are  best  improved  through  dialogue  and  discovery  learning. 

Goals:  To  teach  skills  ("target  abilities")  needed  for  a  wide  variety  of  intellectual  tasks. 
Includes  creative-thinking  skills,  but  emphasis  is  on  reasoning. 

Methods  and  materials:  Ninety-nine  forty-five  minute  lessons  organized  into  six  major 
themes.  Students  have  three  or  four  lessons  per  week  over  a  period  of  two  years. 
Lessons  include  dialogue  and  written  exercises. 

Thinking  skills:  The  authors  assume  that  intellectual  performance  depends  on  abilities, 
methods,  knowledge,  and  attitudes.  These  are  organized  into  six  categories  of  training: 
Foundations  of  Reasoning  -  observation  and  classification,  ordering,  hierarchical 
classification,  analogies,  spatial  reasoning  and  strategies. 
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Understanding  Language  -  word  relations,  language  structure,  reading  comprehension. 
Verbal  Reasoning  -  assertions,  arguments. 

Problem  Solving  -  linear  representations,  tabular  representations,  representations  by 
simulation  and  enactment,  systematic  trial  and  error,  thinking  out  implications. 
Decision  Making  -  the  nature  of  decisions,  using  information,  analyzing  complex 
situations. 

Inventive  Thinking  -  analyzing  and  improving  designs,  procedures  as  designs. 

Targeted  population:  grades  4  to  6. 

Expected  outcome(s):  To  teach  students  to  perform  intellectual  tasks  that  require 
prudent  observation,  deductive  reasoning,  careful  use  of  language,  inferential  use  of 
information,  hypothesis  generation  and  testing,  problem  solving,  inventiveness  and 
creativity,  decision  making,  and  related  skills. 

Promotion  of  independent  generalized  learning:  After  a  teacher  demonstration,  students 
are  encouraged  to  use  the  materials  independently.  In  some  lessons  there  are  activities 
called  "challenges,"  where  students  are  asked  to  apply  a  strategy  on  a  process  in  an  out- 
of-school  context. 

Example:  In  a  lesson  called  Understanding  the  Author’s  Message,  students  can  better 
understand  written  material  if  infer  deduce  the  author’s  purpose  in  writing  it.  The  lesson 
is  therefore  designed  to  give  students  instruction  and  practice  in  inferring  an  author’s 
intentions  in  different  kinds  of  text. 

Efficacy  evidence:  The  results  of  a  study  done  in  Venezuela  show  increased  student 
achievement  on  tests  of  thinking  and  on  standardized  tests.  For  example,  in  the  related 
Project  Intelligence  program  a  group  receiving  the  instruction  scored  better  on  task 
measures  such  as  number  of  solution  features  and  amount  of  detail  (Perkins,  1984). 


Conceptual  Infusion  (Swartz,  1987) 

Assumptions:  Good  thinking  can  be  attributed  to  the  components  of  skills,  activities,  and 
dispositions.  These  component  processes  are  useful  in  various  natural  thinking  contexts. 
Usually,  a  mix  of  these  components  go  together  in  strategic  ways  to  yield  good  thinking. 

Goals: 

1.  To  emphasize  specific  skills,  activities,  and  dispositions  as  educational  goals. 

2.  To  teach  these  goals  in  natural  thinking  contexts. 

3.  To  teach  strategies  for  using  these  skills,  activities,  and  dispositions  in  these  contexts. 

4.  To  find  natural  thinking  contexts  in  the  present  curriculum. 

Methods  and  materials:  Science  textbooks  that  include  thought-provoking  situations 
followed  by  a  series  of  prompts  in  the  form  of  specific  questions  raised  in  sequence 
according  to  the  instructional  model. 

Thinking  skills:  Conceptualizing  alternative  possibilities,  looking  for  good  evidence, 
the  need  to  rule  out  competing  hypotheses,  and  collecting  and  assessing  evidence. 
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Targeted  population:  elementary  and  high  school. 

Expected  outcome(s):  The  desired  result  is  . .  reasonable  reflective  thinking  directed 
at  deciding  what  to  believe  or  do"  (Ennis,  1987,  p  10).  Examples  include  engaging  in 
well-founded  forms  of  intelligence,  judging  the  reliability  and  accuracy  of  the  information 
on  which  such  inferences  are  based,  and  exercising  clarity  and  precision  in  the  way  the 
thoughts  are  conceptualized. 

Promotion  of  independent  generalized  learning:  Students  are  encouraged  to  think  of 
and  describe  some  examples  in  their  own  life  in  which  they  would  like  to  find  out  why 
something  happened. 

Example:  Students  are  asked  to  list  all  possible  explanations  for  a  high  blood  pressure 
reading.  Then,  students  are  asked  to  suggest  a  way  that  each  of  these  explanations  could 
be  tested. 

Efficacy  evidence:  None  given  in  sources  reviewed. 


Jasper  Series  Cognition  and  Technology  Group  at  Vanderbilt  (1992). 

Assumptions:  Based  on  the  constructivist  position  of  learning,  students  should  be  active 
in  the  construction  of  knowledge  rather  then  mere  receivers  of  transmitted  material,  and 
students  need  repeated  opportunities  to  engage  in  in-depth  exploration,  assessment,  and 
revision  of  their  ideas  over  extended  periods  of  time. 

Goals:  Students  are  motivated  to  think  and  reason  about  important,  complex  problems. 

Methods  and  materials:  Anchored  instruction  programs  are  used.  Anchored  instruction 
is  instruction  in  the  context  of  meaningful  problem  solving  environments.  These 
"macrocontexts"  allow  students  to  formulate  and  solve  a  set  of  interconnected 
subproblems.  Videotaped  adventures  of  a  central  character  are  used  (15-20  minutes) 
along  with  other  multimedia  software. 

Thinking  skills:  Students  are  afforded  opportunities  to  solve  interconnected  problems. 
Students  who  are  relative  novices  in  an  area  are  allowed  to  experience  some  of  the 
advantages  of  experts  when  they  are  trying  to  learn  new  information  about  their  area. 
The  approach  is  designed  to  first  have  the  novice  use  the  available  knowledge  to  attempt 
to  understand  the  phenomena  and  activities  depicted  in  an  anchor  and  to  then  be  able 
to  experience  the  changes  in  their  own  attention  and  understanding  as  they  are 
introduced  to  concepts  and  theories  that  are  relevant  to  the  anchors. 

Targeted  population:  late  elementary  age  to  early  high  school. 

Expected  outcome(s):  Students  who  are  more  generative  learners  and  more  self-directed 
rather  than  teacher  directed. 

Promotion  of  independent  generalized  learning:  Students  work  cooperatively,  but  with 
little  intervention  from  the  teacher.  One  major  goal  is  to  foster  self-directed  learning. 
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Example:  In  one  adventure,  the  central  character  and  his  friends  are  faced  with  the 
challenge  of  rescuing  a  wounded  eagle  from  a  dense  forest.  The  student  is  given 
information  which  creates  quite  a  complex  problem  where  many  factors  must  be 
considered. 

Efficacy  evidence:  Students  showed  improvement  from  the  beginning  to  the  end  of  the 
year  in  word  problems  and  planning  skills  (Cognition  and  Technology,  1992).  Jasper 
students  increased  scores  on  the  planning  challenge  by  about  20  points,  while  the  control 
students  raised  their  scores  by  6  points.  On  the  subgoal  of  comprehension,  Jasper 
students  increased  by  24  points,  while  the  control  students  gained  11  points.  They  also 
showed  significant  improvements  in  their  attitudes  towards  problem  solving. 


Relation  of  Skills  Among  Existing  Programs 

The  content  of  the  instructional  programs  was  reviewed  to  determine  whether  they 
included  specific  skills  that  would  be  consistent  with  the  evolving  requirements  of  the  Practical 
Thinking  curriculum.  These  specific  skills  were  determined  independently  from  this  review  and 
were  compared  to  the  skills  and  attitudes  contained  in  the  reviewed  programs.  (See  FaUesen, 
Michel,  Lussier  &  Pounds,  in  preparation,  for  the  requirements.)  The  requirements  were 
identified  based  on  observation  of  weaknesses  in  tactical  decision  making  processes.  Among 
other  limitations,  tactical  decision  making  suffers  from  mind-sets,  using  one  standard  procedure 
regardless  of  situational  differences,  being  unaware  of  critical  assumptions,  using  weak  or 
rationalized  support  for  assertions,  and  lack  of  experience  (Fallesen,  1993).  The  extent  to  which 
the  skills  implied  in  these  deficiencies  are  included  in  the  reviewed  programs  suggests  a 
convergence  of  the  important  skills.  Also  skills  that  are  hypothesized  to  exist  and  are  not 
included  in  any  of  these  reviewed  programs  indicate  where  new  ground  can  be  broken. 

There  were  five  topics  or  themes  from  the  Practical  Thinking  lessons  against  which  the 
cognitive  instruction  programs  were  reviewed.  The  first  topic  was  on  multiple  perspectives,  the 
ability  or  tendency  to  switch  how  one  looks  at  a  problem.  Taking  a  different  perspective  is 
useful  to  generate  different  understandings  of  the  problem,  create  solutions,  and  assess 
solutions.  Different  perspective  includes  being  able  to  force  a  different  perspective  when  the 
problem  solver  is  "stuck"  or  to  consider  various  perspectives  to  survey  many  possible  problem 
and  solution  states.  The  second  topic,  adapting  to  situations,  is  the  tendency  to  match  one’s 
capabilities  to  how  they  respond  to  a  situation  or  a  problem.  Adapting  to  situations  relies  on 
thinking  about  thinking  (metacognition)  to  achieve  focus  and  efficiency.  The  third  lesson  topic 
is  about  finding  hidden  assumptions.  Hidden  assumptions  are  those  aspects  of  a  situation  that 
are  not  apparent  as  assumptions.  Hidden  assumptions  unnecessarily  narrow  one’s  outlook,  and 
can  lead  to  inappropriate  solutions.  Practical  reasoning  is  the  fourth  lesson  topic  and  covers 
informal  reasoning  that  people  use  in  everyday  problem  solving.  The  objective  of  the  lesson  is 
to  convey  standards  of  reasoning  (specifically,  fairness,  relevance,  evidence,  clarity,  and 
consistency)  and  to  give  examples  of  poor  reasoning  to  avoid.  The  fifth  topic  on  integrative 
thinking  illustrates  different  levels  of  thinking  complexity  and  implies  the  characteristics  of 
putting  the  "big  picture"  together. 

Table  4  presents  how  the  cognitive  instruction  programs  correspond  to  the  five  Practical 
Thinking  themes.  All  the  programs  except  Instrumental  Enrichment  were  related  to  three  or 
more  of  the  themes. 
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Table  4. 

Relevant  cognitive  skills  from  reviewed  programs  for  Practical  Thinking. 


21 


Table  4.  (continued) 


22 


Twelve  of  the  fifteen  programs  had  skills  related  to  multiple  perspectives  and  adapting  to 
situations.  The  previous  cognitive  instruction  programs  covered  understanding  and  overcoming 
barriers  to  ideas.  Some  programs  advocated  taking  new  perspectives,  different  perspectives, 
many  perspectives,  and  broad  ones.  Also  the  programs  addressed  considering  other  strategies 
for  how  problems  are  solved  or  decisions  made.  The  reviewed  programs  also  addressed 
adapting  to  the  situation  by  promoting  reflection  and  understanding  of  how  individuals 
themselves  think.  The  programs  covering  adaptation  also  covered  skills  of  organizing  and 
persistence  for  dealing  with  problems. 

Finding  hidden  assumptions  was  covered  in  seven  programs  by  showing  ways  to  identify 
or  recognize  contradictions,  inconsistencies,  and  unstated  or  implicit  assumptions.  Part  of 
finding  hidden  assumptions  involves  challenging  accepted  beliefs  and  knowledge. 

Eleven  of  the  programs  related  to  practical  reasoning.  They  covered  checking  the 
assessment  of  evidence,  clarity,  reliability,  accuracy,  validity,  consistency,  and  plausibility.  In 
some  cases  they  addressed  the  logic  of  conclusions,  screening  of  hypotheses,  and  going  beyond 
simple  solutions.  Eight  of  the  programs  related  to  elements  of  integrative  thinking.  These  dealt 
with  resolving  conflicts,  generating  hypotheses,  making  inferences  and  predictions,  identifying 
connections  or  relationships  among  information. 

Although  there  were  relations  in  topics,  it  is  suspected  that  there  are  subtle  differences 
in  the  intention  and  content  of  the  existing  cognitive  programs  and  those  of  the  Practical 
Thinking  curriculum.  For  example,  while  some  of  the  previous  programs  covered  reasoning, 
they  tended  to  do  so  from  a  logical  standpoint.  The  Practical  TTiinking  requirements  noted  the 
limitations  of  a  logical  approach,  and  chose  to  extend  the  lessons  to  everyday  and  informal  logic. 
SimUar  differences  are  suspected  on  other  topics,  partly  due  to  the  age  of  some  of  the  programs 
and  the  advances  that  have  been  made  in  cognitive  theories.  Also  the  differences  could  be 
partly  due  to  the  younger  audiences  to  which  most  of  the  programs  were  targeted. 


Summary  of  Cognitive  Instruction  Programs 

These  programs  do  not  have  an  overwhelming  research  and  evaluation  base  to  support 
their  use  (Idol  &  Jones,  1991),  but  results  are  aU  in  the  positive  direction.  There  was  no 
empirical  evidence  reported  for  four  of  the  programs  (Heuristic  Model,  IDEAL,  Patterns  of 
Problem  Solving,  and  Conceptual  Infusion),  but  they  appeared  to  be  carefully  developed  and  had 
promising  features.  Other  programs  had  subjective  reports  of  positive  results.  Intelligence 
Applied  indicated  qualitative  improvements.  Critical  Thinking  in  American  History  (CTAH) 
showed  improvement  in  attitudes  about  knowledge.  The  Cognitive  Research  Trust  Thinking 
(CORT)  led  to  improved  attention  to  ideas  and  anecdotal  reports  for  better  perceptual  thinking. 

Some  of  the  programs  showed  improved  component  measures,  but  no  noticeable  effect 
on  overall  problem  solving.  The  Productive  Thinking  Program  had  improvement  indicated  by 
greater  fluency,  better  ideas,  recognition  of  anomalies,  but  not  more  or  better  solutions. 

Heuristic  instruction  showed  improvements  in  mathematical  problem  solving.  Odyssey  resulted 
in  gains  in  the  number  of  solution  features  considered  and  the  amount  of  detail.  Philosophy  for 
Children  resulted  in  improved  reading  skills.  Practicum  in  Thinking  showed  improvements  in 
working  with  others  and  generating  new  ideas. 

Some  programs  did  find  positive  results  on  attitudes  and  overall  performance.  Structure 
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of  Intellect  had  positive  results  on  attitudes  and  performance  and  improved  arithmetic 
performance  on  the  SAT.  The  Jasper  Series  had  positive  results  as  indicated  by  better  scores  on 
word  problems  and  planning  tasks.  The  Instrumental  Enrichment  program  offers  a  considerable 
amount  of  support  for  performance  and  attitude  improvement. 

The  trend  in  data  is  generally  positive  but  difficult  to  fit  into  a  single  coherent  picture. 
Nickerson  (1984)  points  out  that 

"educational  evaluation  is  inherently  difficult,  and  its  results  are  seldom 
unequivocal;  program  developers  have  sometimes  been  sufficiently  convinced  of 
the  merits  of  their  approach  that  they  have  not  been  motivated  to  attempt  an 
evaluation  themselves.  .  .  .  Quantitative  data  on  a  few  programs  indicate  that  they 
produce  modest  improvements  in  performance  on  a  variety  of  tests  of  mental 
ability.  They  make  it  clear  that  no  one  can  yet  assure  the  development  of 
effective  thinking  skills  in  the  classroom,  but  they  reinforce  the  conviction  that 
the  goal  is  a  reasonable  one  and  that  progress  is  being  made  in  its  pursuit."  (p 
36) 

The  determination  of  generalizability  to  specific  career  field  or  jobs  is  even  less  clear, 
because  of  the  lack  of  application,  difficulty  of  measurement  and  failure  to  track  for  any  amount 
of  time.  Four  of  the  programs  did  not  provide  information  on  the  generalizability  or  transfer  of 
the  skills.  Several  of  the  programs  made  it  an  explicit  point  to  encourage  transfer,  but  did  not 
mention  how  that  was  done  (Cognitive  Research  Trust  Thinking,  IDEAL,  Intelligence  Applied, 
and  Odyssey). 

Most  interesting  are  the  techniques  used  by  those  programs  which  required  actual 
practice  of  the  transfer  of  the  skills.  In  the  Philosophy  for  Children  program,  practice  exercises 
emphasized  the  application  of  the  thinking  skills  to  published  material  encountered  in  everyday 
materials,  such  as  advertisements  and  letters  to  the  editor.  Instrumental  Enrichment  provides 
bridging  activities  for  transfer.  Critical  Thinking  in  American  History  encouraged  classroom 
debates  and  individually  prepared  essays  to  form  and  present  the  student’s  own  views.  The 
Conceptual  Infusion  approach  has  the  students  think  of  examples  when  they  could  use  the  skills, 
e.g.  to  find  out  why  something  happened.  Practice  is  encouraged  in  the  Jasper  Series  by  having 
the  students  work  on  similar  problems  and  extensions  of  the  problems.  The  additional  sessions 
focus  on  developing  flexible  knowledge  structures  and  to  make  connections  between  the  tasks  in 
the  video  scenarios  and  historical  events.  Having  the  students  apply  the  skills  to  everyday  or 
personal  situations  seems  like  a  good  technique  to  reinforce  the  skills. 

The  differences  in  context  must  be  considered  to  use  the  previous  cognitive  programs  for 
a  new  course  for  Army  leaders  on  Practical  Thinking.  One  clear  difference  is  the  target 
population.  Five  of  the  programs  target  at  least  up  to  college  students,  while  only  three 
programs  (Structure  of  Intellect,  Instrumental  Enrichment,  and  IDEAL)  include  adults  in  their 
target  audiences.  Most  of  the  programs  have  been  focused  on  a  generally  younger  population, 
and  none  of  those  reviewed  relate  to  specific  jobs.  This  increases  the  importance  of  identifying 
appropriate  skills,  materials,  and  results  with  an  entirely  different  target  population. 

The  time  available  for  classroom  instruction  of  Army  leaders  is  under  more  severe 
demands  than  the  general  curricula  of  most  of  these  applications.  Most  of  the  reviewed 
programs  require  fairly  extensive  class  room  time  over  an  extended  period.  The  Instrumental 
Enrichment  program  cites  a  minimum  of  80  hours  over  a  1  to  3  year  period,  and  it  takes  about 
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100  hours  before  improvements  are  noticed.  The  college  programs  (Patterns  of  Problem  Solving, 
Practicum  in  Thinking,  and  Intelligence  Applied)  run  from  about  40  to  100  hours  of  class  time. 
Considering  that  typical  schooling  falls  into  15  to  30  hours  of  classroom  contact  per  week  with 
many  topics  to  cover,  the  cognitive  instruction  programs  make  considerable  demands  on 
schedules  in  civilian  applications.  These  rather  lengthy  periods  of  instruction  point  to  the  need 
to  develop  a  very  efficient  curriculum. 

In  general,  there  is  a  tremendous  amount  of  variety  among  the  programs  with  regard  to 
the  thinking  skills  advocated  by  each.  They  tend  to  cover  a  considerably  large  number  of 
general  thinking  skills.  The  variety  of  general  skills  and  specific  skills  that  are  targeted  by  these 
programs  lend  credibility  to  the  selection  of  skills  and  teaching  points  upon  which  the  Practical 
Thinking  instruction  for  Army  leaders  was  based. 
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